Many deep neural networks trained on natural images exhibit a curious phenomenon in common: on the first layer they learn features similar to Gabor filters and color blobs. Such first-layer features appear not to be specific to a particular dataset or task, but general in that they are applicable to many datasets and tasks. Features must eventually transition from general to specific by the last layer of the network, but this transition has not been studied extensively. In this paper we experimentally quantify the generality versus specificity of neurons in each layer of a deep convolutional neural network and report a few surprising results. Transferability is negatively affected by two distinct issues: (1) the specialization of higher layer neurons to their original task at the expense of performance on the target task, which was expected, and (2) optimization difficulties related to splitting networks between co-adapted neurons, which was not expected. In an example network trained on ImageNet, we demonstrate that either of these two issues may dominate, depending on whether features are transferred from the bottom, middle, or top of the network. We also document that the transferability of features decreases as the distance between the base task and target task increases, but that transferring features even from distant tasks can be better than using random features. A final surprising result is that initializing a network with transferred features from almost any number of layers can produce a boost to generalization that lingers even after fine-tuning to the target dataset.

Among the many things rodents have taught neuroscientists is that, in a region called the hippocampus, the brain creates a new map for every unique spatial context -- for instance, a different room or maze. But scientists have so far struggled to learn how animals decide when a context is novel enough to merit creating, or at least revising, these mental maps. In a study in eLife, MIT and Harvard University researchers propose a new understanding: The process of "remapping" can be mathematically modeled as a feat of probabilistic reasoning by the rodents. The approach offers scientists a new way to interpret many experiments that depend on measuring remapping to investigate learning and memory. Remapping is integral to that pursuit, because animals (and people) associate learning closely with context, and hippocampal maps indicate which context an animal believes itself to be in.

Many deep neural networks trained on natural images exhibit a curious phenomenon in common: on the first layer they learn features similar to Gabor filters and color blobs. Such first-layer features appear not to be specific to a particular dataset or task, but general in that they are applicable to many datasets and tasks. Features must eventually transition from general to specific by the last layer of the network, but this transition has not been studied extensively. In this paper we experimentally quantify the generality versus specificity of neurons in each layer of a deep convolutional neural network and report a few surprising results. Transferability is negatively affected by two distinct issues: (1) the specialization of higher layer neurons to their original task at the expense of performance on the target task, which was expected, and (2) optimization difficulties related to splitting networks between co-adapted neurons, which was not expected.

Digitization and automation are ever-growing topics in relation to the workplace. A famous Oxford study on the future of employment from 2013 estimated that up to 47% of American jobs may be automated by 2035; a brand new McKinsey study shows that current technologies could automate 45 percent of job activities; and the business mantra goes that if you can digitize, you should digitize to gain a competitive advantage. But how do we, as human beings, really feel about potentially working with or even for AIs, and what impact do we think they will have on our workplace? A recent study conducted in the US, UK and Denmark explores people's openness towards working with and for "unbiased computer programs"--defined as "a software robot that makes decisions or proposals for decisions based on data from HR, financial or market information. The software robot is unbiased, i.e. it is not affected by the personal, social and cultural bias that influence human decision making, but balances all input only based on the data."

In AAAI-80, pp. 102–104.